Human Perception of Statistical Charts: An Introduction to Graphical Testing Methods

Pepperdine University

Emily Robinson

Cal Poly - San Luis Obispo

Reka Howard

University of Nebraska - Lincoln

Susan VanderPlas

University of Nebraska - Lincoln

Outline

Motivation and Background

Comprehensive Graphical Testing

Research Goals

Perception through Lineups

Prediction through ‘You Draw It’

Numerical Translation and Estimation

Overall Conclusions and Discussion

Introduction to Graphics

Data visualization is defined as the art of drawing graphical charts in order to display data Unwin (2020).

What are graphics useful for? Lewandowsky and Spence (1989)

📉 Data cleaning.

🔍 Exploring data structure.

💬 Communicating information.

Who uses graphics?

History of Graphics

Visit the Timeline of Infographics by RJ Andrews (Info We Trust Data Storyteller).

Grammar of Graphics (Wilkinson 2012)

Graphics are viewed as a mapping from variables in the data set to visual attributes on the chart.

Building a masterpiece, by Allison Horst

Testing Statistical Graphics

Evaluate design choices and understand cognitive biases through the use of visual tests.

Could ask participants to:

📉 identify differences in graphs.

📖 read information off of a chart accurately.

🌍 use data to make correct real-world decisions.

✏️ predict the next few observations.

Task complexity

Carpenter and Shah (1988) identifies pattern recognition, interpretative processes, and integrative processes as strategies and processes required to complete tasks of varying degrees of complexity.

  • Pattern recognition requires the viewer to encode graphic patterns.

  • Interpretive processes operate on those patterns to construct meaning.

  • Integrative processes then relate the meanings to the contextual scenario as inferred from labels and titles.

Lineup Protocal (Buja et al. 2009)

Introduction to Visual Inference

When doing exploratory data analysis, how do we know if what we see is actually there?

Lineup Protocol (Buja et. al, 2009)

Embed a target plot (actual data) in a lineup of null plots (randomly permuted data sets).

Introduction to Visual Inference

Lineup Studies 📉 📊 📈

“You Draw It” (Robinson, Howard, and VanderPlas 2023)

“You Draw It” for News Media

New York Times (Aisch, Cox, and Quealy 2015)

“You Draw It” for Graphical Testing

Eye Fitting Straight Lines in the Modern Era (Robinson, Howard, and VanderPlas 2022)

youdrawitR Package

Perception of Logarithmic Scales

Exponential Growth

Von Bergmann (2021)

Benefits and pitfalls of log scales

Burn-Murdoch et al. (2020)

Research objectives

Big Idea: Are there benefits to displaying exponentially increasing data on a log scale rather than a linear scale?

  1. Perception through Lineups – tests an individual’s ability to perceptually differentiate exponentially increasing data with differing rates of change on both the linear and log scale.

  2. Prediction with ‘You Draw It’ – tests an individual’s ability to make predictions for exponentially increasing data.

  3. Estimation by Numerical Translation – tests an individual’s ability to translate a graph of exponentially increasing data into real value quantities.

The series of graphical tests were conducted through an RShiny application found at https://emily-robinson.shinyapps.io/perception-of-statistical-graphics-log/.

Graphical Tasks

Study Participant Prompt: Which plot is most different?

Study Participant Prompt: Use your mouse to fill in the trend in the yellow box region.

Study Participant Prompt: From 4520 to 4540, the population increases by ____ Tribbles [Ewoks].

Study Results

Conclusions

1. Perception through Lineups

  • Perceptual differences result from the contextual appearance (depends on choice of scale) of the trends.

2. Prediction through ‘You Draw It’

  • Clear underestimation of forecasting trends with high exponential growth rates when participants were asked to make predictions on the linear scale.

3. Numerical Translation and Estimation

  • Log logic is difficult and that we often misinterpret and miscalculate multiplicative reasoning.
  • Estimation accuracy for small magnitudes was improved by the use of the log scale, but sacrifices in accuracy on the log scale became apparent as magnitudes increased leading to advantages on the linear scale.

Overall Recommendations

  • Perceptual advantages of the use of log scales due to the change in contextual appearance.

  • Our understanding of log logic is flawed when translating the information into context.

  • We recommend consideration of both user needs and graph specific tasks when presenting data on the log scale.

  • Caution should be taken when interpretation of large magnitudes is required, but advantages may appear when it is necessary to visually identify and interpret small magnitudes on the chart.

References

Aisch, Gregor, Nate Cohn, Amanda Cox, Josh Katz, Adam Pearce, and Kevin Quealy. 2016. “Live Presidential Forecast.” The New York Times. The New York Times. https://www.nytimes.com/elections/2016/forecast/president.
Aisch, Gregor, Amanda Cox, and Kevin Quealy. 2015. “You Draw It: How Family Income Predicts Children’s College Chances.” The New York Times. The New York Times. https://www.nytimes.com/interactive/2015/05/28/upshot/you-draw-it-how-family-income-affects-childrens-college-chances.html.
Buja, Andreas, Dianne Cook, Heike Hofmann, Michael Lawrence, Eun-Kyung Lee, Deborah F Swayne, and Hadley Wickham. 2009. “Statistical Inference for Exploratory Data Analysis and Model Diagnostics.” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 367 (1906): 4361–83.
Burn-Murdoch, John, Caroline Nevitt, Cale Tilford, Andrew Rininsland, Joanna Kao, Oliver Elliott, Emma Lewis, Brooke Fox, and Martin Stabe. 2020. “Coronavirus Tracked: Has the Epidemic Peaked Near You?” Coronavirus Chart: See How Your Country Compares. Financial Times. https://ig.ft.com/coronavirus-chart/?areas=eur.
Carpenter, Patricia A, and Priti Shah. 1988. “A Model of the Perceptual and Conceptual Processes in Graph Comprehension,” 26.
Chandar, Nandini, Deirdre Collier, and Paul Miranti. 2012. “Graph Standardization and Management Accounting at AT&t During the 1920s.” Accounting History 17 (1): 35–62.
Gouretski, Viktor, and Klaus Peter Koltermann. 2007. “How Much Is the Ocean Really Warming?” Geophysical Research Letters 34 (1).
Harms, Hans. 1991. “August Friedrich Wilhelm Crome (1753-1833) Autor Begehrter Wirtschaftskarten.” Cartographica Helvetica 3: 33–38.
Lewandowsky, Stephan, and Ian Spence. 1989. “The Perception of Statistical Graphs.” Sociological Methods & Research 18 (2-3): 200–242. https://doi.org/10.1177/0049124189018002002.
Loy, Adam, Lendie Follett, and Heike Hofmann. 2016. “Variations of q–q Plots: The Power of Our Eyes!” The American Statistician 70 (2): 202–14.
Majumder, Mahbubul, Heike Hofmann, and Dianne Cook. 2013. “Validation of Visual Statistical Inference, Applied to Linear Models.” Journal of the American Statistical Association 108 (503): 942–56.
———. 2014. “Human Factors Influencing Visual Statistical Inference.” arXiv Preprint arXiv:1408.1974.
Playfair, W. 1801. “The Statistical Breviary; Shewing, on a Principle Entirely New, the Resources of Every State and Kingdom in Europe, Wallis, Londres.” Press, Chicago.
Robinson, Emily A, Reka Howard, and Susan VanderPlas. 2022. “Eye Fitting Straight Lines in the Modern Era.” Journal of Computational and Graphical Statistics, 1–8.
———. 2023. ‘You Draw It’: Implementation of Visually Fitted Trends with R2d3.” Journal of Data Science 21 (2): 281–94.
Unwin, Anthony. 2020. “Why Is Data Visualization Important? What Is Important in Data Visualization?” Harvard Data Science Review, January. https://doi.org/10.1162/99608f92.8ae4d525.
VanderPlas, Susan, and Heike Hofmann. 2015. “Spatial Reasoning and Data Displays.” IEEE Transactions on Visualization and Computer Graphics 22 (1): 459–68.
———. 2017. “Clusters Beat Trend!? Testing Feature Hierarchy in Statistical Graphics.” Journal of Computational and Graphical Statistics 26 (2): 231–42.
VanderPlas, Susan, Christian Röttger, Dianne Cook, and Heike Hofmann. 2021. “Statistical Significance Calculations for Scenarios in Visual Inference.” Stat 10 (1): e337.
Von Bergmann, Jens. 2021. “Xkcd_exponential: Public Health Vs Scientists.” mountainMath. GitHub. March 17, 2021. https://github.com/mountainMath/xkcd_exponential.
Walker, Francis Amasa. 2013. “Statistical Atlas of the United States Based on the Results of the Ninth Census 1870 with Contributions from Many Eminent Men of Science and Several Departments of the Government.” [New York] J. Bien, lith.
Wilkinson, Leland. 2012. The Grammar of Graphics. Springer.
Yates, JoAnne. 1985. “Graphs as a Managerial Tool: A Case Study of Du Pont’s Use of Graphs in the Early Twentieth Century.” The Journal of Business Communication (1973) 22 (1): 5–33.

Emily A. Robinson

github.com/earobinson95